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1.  Introduction.  Let  W(t),  t  >  0,  W(0)  -  0  be  a  standard  Wiener 

process.  Independent  of  N(t),  t  >  0,  H(0)  -  0  a  Poisson  process  with 

(constant)  unit  jumps,  and  EN(t)  ■  Xt,  t  >  0.  Let  their  signs  fields  be 

f  (t)  -  9(W(s),  0  <  s  <  t)  and  6(t)  -  9(N(s),  0  £  a  £  t).  Let  g(y)  bo 

a  function  and  L(g(y))  a  differential  operator  on  g,  for  example, 

•  2 

L(g(y>)  -  a(y)  ^  g(y)  +  d(y)  g(y)  +  c(y)g(y). 

ay  ay 

Let  X(t,y)  be  a  stochastic  process  depending  on  t  >  0  and  a  parameter 
y,  such  that  (for  m  Xt), 

(1.1)  Xt(t,y)  -  L(X(t,y))  +  u(X(t,y),y)  +  ^dt^* 

X(0.y)  -  g(y)  m  x. 

with  white  noise,  incremental  Poisson  jump  process  and  where 

u(X(t,y),y)  la  measurable  with  respect  to  9(P(t)  U  6(t))  (l.e.  u  Is 
non-anticipative)  and  satisfies,  for  ▲  a  constant,  and  B  >  0  a 
constant, 

(1.2)  | u— Aj  £  B, 

all  0<  t£T,  0  <  T  <  .  a  constant.  The  cost  function  for  a  given  u 

satisfying  (1.2)  Is,  for  9>  0  a  constant, 

X 

(1.3)  J(u,y)  -  [  e“a*  B(X2(u,s))ds. 


Ibt  object  is  to  characterize  tha  optimal  u  for  which  J  ia  minimised 
The  cases  T  <  •  and  T  ■  ■  are  treated  separately.  The  method  employs  a 
suitable  Bellman  equation,  a  maximum  principle  for  parabolic  partial 
differential~difference  equations  and  the  Ito  rule.  The  method  follows  [4] . 


2.  Finite  Interval  Control. 

Let  T  <  «*.  Define,  for  0  <;  t  <  T, 


(2.1) 


m  V(»it,y)  -  inf  f  e"**  S(X2(s,y))ds 
ju-Aj  <  B  J0 


and  X(0,y)  ■  g(y)  -  x. 
pt  h  pt+fa  t+b 

By  writing  m  +  -  !  ,  heuristic  arguments  ([2],  pp.  179*180) 

J0  J0  Jh  Jt 
yield  a  Bellman  equation 

(2.2)  I2  +  Inf  (uV  )  +  L(,(y))V  +  ij  -  aV  -  V 

x  x  “ 

+  X(V(xfl,t,y)  *  V(x,t,y))  -  0, 

with  u  ■  u(x). 

On  heuristic  grounds,  a  solution  to  (2.2)  is  sought  such  that 


(2.3) 


x2  +  [<A-B)  +  L(g(y))]Vx  +  \  -  «rV-Vt 


+  X(V(x^l,t,y)  -  V(x,t,y))  -  0 


(2.4) 


for  Vx  >  0,  x  >  b(t,y) 


x2  +  I(AfB)  +  L(g(y)))Vx  +  \  -  aV-Vfc 


+  X(V(*+l,t,y)  -  V(x,t,y))  -  0 


for  V  <  0,  x  <  b(t,y) 


The  object  is  to  characterize  the  optimal  u  for  which  J  la  minimized. 
The  cases  T  <  m  and  T  ■  *  are  treated  separately.  The  method  employs  a 
suitable  Bellman  equation,  a  maximum  principle  for  parabolic  partial 
differential-difference  equations  and  the  Ito  rule.  The  method  follows  (4j . 


2.  Finite  Interval  Control. 

Let  T  <  •.  Define,  for  0  £  t  <  T, 


(2.1) 


I 


V  m  V(x*t,y)  -  inf 

|u-A|  <  B  J0 


X(O.y)  -  g(y)  -  x. 
.h  _t+h  _t+h 


E(X2(s,y))ds 


pt  h  pt+h  Wfc 

By  writing  I  m  +  -  !  ,  heuristic  arguments  ([2],  pp.  179-180) 

J0  J0  Jh  Jt 


yield  a  Bellman  equation 


(2.2)  x2  +  inf  (uV  )  +  L(g(y))V  +  |  V-  aV  -  V. 

|u-Aj  £  B  *  x  2  xx  t 

+  X(V(ipH.t,y)  -  V(x,t,y))  -  0, 

with  u  »  u(x). 

On  heuristic  grounds,  a  solution  to  (2.2)  Is  sought  such  that 


(2.3) 


x2  +  [(A-B)  +  L(g(y))]Vx  +  j*xx-  «V-Vt 


+  X(V(*+l»t,y)  -  V(x,t,y))  -  0 
for  Vx  >  0,  x  >  b(t,y) 

and 

(2.4)  x2  +  [(AfB)  +  L(g(y))]Vx  +  \  V  -  crV-Vt 

+  X(V(x+l,t,y)  -  V(x,t,y))  -  0 
for  Vx  <  0,  x  <  b(t,y) 
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vhtrc  b(t,y)  Is  obtained  from  the  following  conditions,  letting 
V  ■  Vx  in  (2.3)  and  V  ■  V2  in  (2.4)  for  0  <  t  <  T,  all  y: 

Vj(b(t,y),t,y)  -  V2(b(t,y),t,y) 

Vx  x(b(t,y),t,y)  -  V2  x(b(t,y),t,y)  -  0 

(2.5)  V1(x,0,y)  -  V2(x,0,y)  -  0. 

(2.5a)  Vi,*x(b(t»y),t»y)  -  °  * 

For  &  constant,  denote 

(2.6)  J(x,t,y,R)  ■  f  e"**E((R+L(g(y))s  +  W(s)  +  N(s)  +  x)2ds. 

J0 

A  direct  coaqnitation  verifies  that  J(x,t,y,A«B)  ia  a  particular 
solution  to  (2.3)  and  J(x,t,y,  A+B)  is  a  particular  solution  to  (2.4). 

a««.— pMpp  1.  There  is  a  non- zero  solution  H^(x,t,y)  with  H1(x,0,y) 

to  1 

-oR  +  (A-B  +  L(g(y))Hx  +  -j  -  Ht^X(H(x+l,t,y)  -  H(x,t,yJ  -  0 
such  that 

®j(x,t,y)  -  0(e"Px) 

(2.7)  *  0(a‘6*) 

for  acme  p  >  0,  ft  >  0,  all  t,y,  as  x  -*  ». 

Also,  there  is  a  non-sero  solution  H2(x,t,y)  with  H^x.O.y)  ■  0, 

t0  1 

"Of®  +  (A+B  +  L(g(y)))Rx  +  |  -  Hfc 

+  X(R(x+l,t,y)  -  H(x,t,y^  »  0 

such  that 


H2(*.t»F)  -  0(a4x) 


V*  . 


(2.8)  Ha  (xtt,y)  -  0(exx) 


for  som  6  >  0,  X  >  0»  «11  t»y»  as  x 


On®  say  eh«n  writ® 


(2.9)  Vl(x,t,y)  -  J(x,t,y,  A-B)  +  Hj^  (x,t,y) 

(2.10)  V2(x,t,y)  •  J(x,t,y,A+Br+ H2(x,t,y) 

ud  assume  b(t,y)  is  determined  from  (2.5),  (2.9),  (2.10). 
This  notivates 


Th®or®a  1  Assum®  th®  conditions  and  results  of  sections  1,2  hold  for 
0  <  T  <  m. 

The  optimal  Uq  may  be  expressed  as 


A-B  if  XQ(t,y)  >  b(T-t,y) 

(2.11)  *»0(X0(t,y),y)  -  < 

AfB  if  X  (t.y)  £  b(T-t,y) 

e  V 

where  b(t,y)  is  obtained  from  (2.5),  (2.9)-(2.10)  and 


(2.12)  *0t(t,y)  -  L(Xfl(t,y))  +  u0  (XQ(t,y),y)  + 


+  M£1 

dt 


with  Xo(0,y)  -  g(y)  m  x. 


Ut  D  m  V  . 


From  (2.3),  (2.4)  (omitting  (t,y)  arguments) 


(2.13)  <A±B  +  L(g(y)))  \  -«D-Dt-xD  -  -2  -xD(x+l) 

where  the  argument  on  the  left  side  of  (2.13)  la  x. 


V 


Fro.  (2.6), 


(2.14)  *„(•*«»*»  At®)  >  °- 

Assumption  1,  (2.7)- (2.10),  (2.14),  imply  that 

(2.15)  Vl>Jtx(x,t,y)  >  0,  for  *«Ch  t,y,  ux-». 
and 

(2.16)  V,  (x,t,y)  >  0,  for  each.t:,y,  UX4*». 

For  fixed  (t,y),  (omitting  the  fixed  arguments  (t,y)),  suppose 
there  existed  a  finite  number  r  >  b,  and  a  number  0,  0  <  0  <  1  and  such 

that  using  (2.5a),  - 

(2.17)  D(x)  <0,  b<r-0<x<r,  D(r)  -  0, 

and 

(2.17)  D(x)  >0,  x  >  r, 

using  (2.15).  It  follows  that  for  x  >  r-0,  the  left  side  of  (2.13) 
is  negative.  By  ([1],  Lemma  1,  p.34),  D  cannot  have  a  negative  minimum 
for  x>  r-0,  a  contradiction  to  (2.17),  since  (2.15)  and  (2.5a)  hold. 

Hence  D  >  0  for  x  >  b. 

For  b-1  <  x  <  b,  a  similar  argument  using  (2.16)  and  that  D(x)  >  0  for 
b  <  x  <  b  +  1  yields  that  D(x)  >  0  for  b-1  <  x  <  b.  Continuing  by  Iteration, 
D(x)  >  0  for  x  <  b,  so  that  D  >  0  for  all  (x,t,y). 

Al»°»  ^x(®(t))  •  0  by  (2.5),  implying  that  (2. 3), (2.4)  (2.9),  (2.10) 
is  a  solution  to  the  Bellman  equation  (2.2).  To  show  that  u^  is  optimal, 
define 

(2.19)  K(X(t,y),t,y)  a  V(X(t,y),T-t,y)a“art 

for  0  £  t  £  T. 

Noting  that  K(X(0,y),0,y)  •  V(x,T,y)  and  K(X(T,y) ,T,y)  -  0,  the  Ito 
ytaWs  that  ([2] ,  pp.  125-126)  for  a  u  and  corresponding  X(t,y), 


(2.20)  •  X  (s,y)ds  -  V(x,T,y) 

•  0 


f  •"or*(-arV(X(s,y),T-s,y)  -  V  (X(s,y),T-s,y) 

J0  C 

+  inf  (u(X(s,y) ,y)  V  (X(s,y),T-s,y)) 

|u-A!  <  B 

+  |  VB(2(«,y),T-«,y)  +  L(g(y»  Vx(X(«,y),T-s,y)  +  X2(s,y))ds 


fT  -« 

+  J  •  (V(X(8,y),T-«,y)d»(3) 

*  0 

T 

+  J  •~a9Vx(Z(»,y),T-a,y)M(9) 


Px  ...... 

+  J  •”®8(u(X(s,y),y)V  (X(s,y),T-s,y)  -  inf  (u(X(s/y),y)V  (X(s,y),T-s,y)) 
J0'  X  |u-A|._^  B  x 

Thn  fourth  integral  on  the  right  of  (2.20)  is  non- negative.  Upon  taking 
expectations  of  (2.20),  the  third  integral  on  the  right  is  zero,  and  one 
obtains  (oaitting  the  arguments  from  the  first  and  fourth  integrals,  and 
combining  the  first  and  second  integrals,  all  on  the  right) 

X 

(2.21)  f  e‘«*E(X2(a,y))da  -  V(x,T,y)  - 
J0 

T 

E  f  e"®* (-crV-V  +  inf  (uV  )  +  L(g(y))V  +  \  V  +  X2 

J  0  z  |  u.^j  <  b  X  x  a  xx 


+  A  (V(X(a)+l)-V)ds 


+  ,  (uVx»d.. 

Jo  IU*AI  £  B 


The  first  integral  on  the  right  of  (2.21)  is  zero  by  (2. 2)- (2. 4),  and 
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Che  second  integral  is  non-negative,  and  is  zero  for  u  ■  Uq,  X  ■  X^. 
Hence  (2.21)  yields  that 
T 

(2.22)  f  e"**  E(X2(s,y))ds  >  V(x,T,y) 

J0 

and 

,T  , 

(2.23)  f  e”®*E(xf (s,y))ds  -  V(x,T,y), 
and  (2.22),  (2.23)  imply  uQ  is  optimal. 

3.  Infinite  Interval  Control 


Assume  the  conditions  of  section  1  hold  for  T  »  •  so  that  (1.3)  is 


now 

(3.1)  J(u,y)  -  f* e"a8E(X2(s,y))ds. 

J0 

Define 


(3.2) 


V  •  V(x,y) 


inf 

!U-A|  <  B 


r.* 


■cr» 


E(X  (s,y))ds 


with  X(0,y)  -  g(y)  ■  x. 


By  writing  j:-£*r  ,  heuristic  arguments  ([2],  pp.  179* 


180) 


yield  a  Bellman  equation 


(3.3)  x2  -crV(x.y)  +  V  (x,y)  +  inf  (u(x,y)V(x,y)) 

2  «  l«-A|S  * 

+  L(g(y))Vjt(x,y)  +  X(V(*+1.7)-V(x,y))  -  0. 

On  heuristic  grounds,  a  solution  to  (3.3)  is  sought  such  that  (omitting 
the  (x,y)  arguments) 
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(3.4)  x2  -  aV  +  -|  Vxjc  +  (A’B  +  L<8<y>»vx  +  X(V(*fl,y)-V)  -  O 

for  Vx  >  0,  x  >  b(y) 

and 

(3.5)  x2  -crV  +  \  Va  +  (A4^L(g(y))Vx  +  X(V(x*l,y)-V)  -  0 

for  Vx  <  0,  x  <  b(y), 

vherp  b(y)  is  obtained  from  the  following  matching  conditions,  letting 
V  ■  Vx  in(3.4)  and  V  ■  V2  in  (3.5) : 

V1(b(y)fy)  -  V2(b(y),y) 

(3.6)  Vlx(b(y),y)  -  V2  x(b(y),y)  -  0 

(3.6a>  v1>xx<b»y>»y>  >°  • 

For  R  constant,  denote 

(3.7)  J(x,y,R)  -  ]V^E((R+L(g(y)))s  +  W(a)+H(s)4*)2ds. 

J0 

A  computation  verifies  that  J(x,y,A-B)  is  a  particular  solution  to 
(3.4)  and  J(x,y,  A+B)  is  a  particular  solution  to  (3.5). 

Assumption  2.  There  Is  a  non-zero  solution  H^(x,y)  to  (omitting  (x,y) 
argument) 

-orB  +  (A-B  +  Mgfr)))!^  +  *|  8^  +  X(H(*rt.y)-H)  -  0 

such  that 

8^(x,y)  -  0(e*pJt) 

(3.8)  HijXX(x,y)  »  0(e“®*) 


for  some  0  >  0,  ft  >  0,  for  each  y,  as  x  -»  Similarly,  there  is 
non-zero  solution  I^C^y)  to 


-aH  +  (A+B  +  L(g(y)))Hx  +  +  \ (H(x+l,y)-H)  -  0 

such  that 

H^x.y)  -  0(e*x) 

(3.9)  H2,xx(x*y)  "  0<*XX) 

for  some  ft  >  0,  X  >  0,  for  each  y  as  x  -*  -a*. 

Then  one  may  write 

(3.10)  V1(x,y)  -  J(x,y,A-B)  +  H^x.y) 

(3.11)  V2(x,y)  -  J(x,y,A+B)  +  H2(xfy) 

and  assume  that  b(y)  is  determined  from  (3.6),  (3.10),  (3.U). 
As  in  Section  2,  one  then  obtains 


Theorem  2.  Assume  that  the  conditions  and  results  of  sections  1,3 
for  T  -  m. 

The  optimal  may  be  expressed  as 


(3.12) 


UjO^t.y) ,  y) 


AfB 


if  Xl(t,y)  >  b(y) 
if  XL(t,y)  <  b(y) 


where  b(y)  is  obtained  from  (3.6),  (3.10)-(3.11)  and 


(3.13) 


xu<t,y>  -  . 

•nd  Xj^O.y)  -  g(y)  ■  x. 

Proof.  Th«  proof  follows  that  of  Theorem  1. 

Let  D  -  V  .  From  (3.4),  (3.5)  (omitting  (x,y)  arguments) 

(3.14)  (A£B  +  L(g(y)))Dx  +  \  -rf>  -  XD  -  -2  -  XD(x«.) 

where  the  argument  on  the  left  side  of  (3.14)  Is  x. 

From  (3.7), 

(3.15)  Jjot(x,y^4fB)  >  0. 

Assumption  2,  (3.8)-(3.11) ,  (3.15)  Imply  that 

(3.16)  V,  __(x,y)  >  0  for  each  t,y,  as  x  -»  «, 
and 

(3.17)  V2,xx^x,y^  >  0  *or  **ch  t,y*  **  **♦ 

By  an  argument  identical  to  that  given  In  the  proof  of  Theorem  1 
using  the  appropriate  maximum  principle  ([1],  Theorem  18,  p.  53),  It 
follows  that  D(x,y)  £0  for  all  (x,y).  Since  Vx(b(y),y)  -  0 


by  (3.6),  it  follows  that  (3.4),  (3.5),  (3.10),  (3.11)  constitute  a 
solution  to  the  Bellman  equation  (3.3).  To  show  that  u^  is  optimal, 
define 


(3.20)  K(X(t,y),y)  -  V(X( t,y),y)e"at 


for  t  >  0. 


Noting  that  K(X(0,y),y)  »  V(x,y),  one  obtains  from  the  Ito  rule 

((2),  pp.  125-126)  for  a  u  and  corresponding  X(t,y), (omit ting  (X(s,y), y) 

arguments  on  the  right  side). 


(3.21) 


f  e“®#  X2(s,y)ds  +  V(X(t,y),y)e**e  -  V(x,y) 
J0 


f  •_as(X2(a,y)  -  «V  +  inf  (uV  )  +  L(g(y))V  +  ■§  V  )ds 
*  0  Ju-A|<B  *  X  2  XX 


+f  •”a*(uV_  -  inf  (uV  ))d» 
Ja  *  | u-Aj  <  B 


0 

.t 


+f  «"a*(V(X(s,y)  ,s)dN(s) 
•  0 


The  second  integral  on  the  right  of  (3.21)  la  non-negative.  Upon  taking 
expectations  in  (3.21),  the  fourth  term  on  the  right  is  aero,  and  combining 
the  first  and  third  terms  on  the  right  yields  that 


(3.22)  rt«’ofSE(X2(s,y))d»  +  E(V(X(t,y),y))«’at-V(*,y)  - 
*  0 

Ef  •"flf,(X2(s.y)-«V  +  inf  (uV  )  +  L(g(y))V  +  \  ¥ 

J0  |u-A|  <  B  2 

+  X(V(X(s)+l,y)-V)ds 
pt 

+  E|  •“a*(uV  -  inf  (uV  ))d*. 

"  0  *  Ju-Aj  <  B  X 

By  (3.3) ,  the  first  tern  on  th«  right  of  (3.22)  is  zero. 

Th«  second  term  on  tbs  right  of  (3.22)  is  non-negative,  and  is  zero  for 
u  ■  u^.  By  Assuaption  2,  (3.7)- (3. 11),  it  follows  that,  for  (x,y)  fixed, 
there  is  a  constant  D  >  0  such  that 

(3.23)  E(V(X(t,y) ,y))e"0ft  <  E(|xl+(|A|+»*-|L(g(y)l ) W-|W(t)J+N(t))2e"art-H)e'ort 
or 

(3.24)  E(V(X(t,y),y)e'®t  <  C(x,y)t2e"®t 
for  sons  positive  constant  C(x,y). 

Letting  t  -*  m  in  (3.22),  by  (3.24)  one  obtains 

(3.23)  f  e~flf*E(X2(a,y))ds  >  V(x,y) 

*0 


(3.26)  J* •”ar*E(X2(s,y))ds  -  V(x,y) , 


so  that  (3.23),  (3.26)  laply  that  is  optimal. 
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4.  Additional  Constraints 

Certain  additional  constraints  may  be  incorporated  and  treated  by 
these  methods.  An  illustrative  example  in  the  case  T  <  «•  is  the  added 
constraint 

(4.1)  E(X2(a)>  -  C 

where  0  <  a  <  T  and  C  >  0.  This  may  be  handled  by  adding  the  condition 

(4.2)  Vt(*,t,y)Itwa-  e~°*C 

to  conditions  (2.5),  and  proceeding  as  before.  See  [3]  for  another  approach. 
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